Many companies maintain data centers that include one or more servers to provide various computer-based services. Although a wide variety of services may be provided, some common examples include electronic shopping services, news services, mapping services, search services, and interpersonal messaging services (such as email). Data centers may include multiple servers to accommodate large computing loads associated with requests for services from clients in a robust manner. Such data centers may also include a load balancer that redirects traffic from servers that are experiencing heavy loads to servers that have greater availability.
Additionally, many servers provide services via a stateless protocol such as hyper text transfer protocol (HTTP). A stateless protocol is a protocol that does not require maintaining state between two entities communicating with the protocol. The benefits of stateless protocols are simplicity of interfaces, scalability of systems implementing the protocol, ubiquity of the protocols (e.g., in the case of HTTP), visibility of communications, portability of communications, and reliability of communications. However, with stateless protocols, the communications may not be natively aware of state. Thus, in order to maintain state associated with communications over a communications protocol, mechanisms that are external to the stateless communications protocol itself are used.
One way state may be maintained in conjunction with a stateless protocol is by using sessions. A session is a series of communications that are considered to be part of a single “conversation” and are stored locally. Communications between entities may utilize a session ID to identify a particular conversation. Servers can store state in order to maintain information about this conversation. In the example in which the service provided is electronic shopping, the state may include a cart that includes items a user has collected.
When used in conjunction with a load balancer, storing session information presents certain issues. Specifically, the task of a load balancer is to redistribute communications among multiple servers in order to balance the load experienced by each server. However, if the load balancer were to change which server services a particular session, from a first server to a second server, then session data would effectively be lost to the server handling the session, since the second server would not store or have access to the session data stored at the first server.
A technique known as “sticky sessions” is typically used to solve this issue. The technique involves preventing a load balancer from changing the server that services a particular session. Thus, ongoing sessions remain with their respective servers and are not affected by the load balancer. Although this technique allows session data to be retained in one server throughout a session, the technique of sticky sessions itself presents certain issues. Specifically, because a load balancer is unable to transfer communications associated with an ongoing session between servers, certain servers may accumulate sessions, and thus accumulate load, while other servers remain relatively free. With sticky sessions active, the load balancer would not be able to redirect traffic for the busier server as long as associated sessions remain active, thus negating the effectiveness of the load balancer. Another related issue is that adding servers to the data center does not immediately have an impact on the load of a particular server. This is because regardless of the number of servers available, traffic associated with ongoing sessions cannot be redirected if these sessions are sticky sessions. A further issue is that if a server becomes inactive, session data associated with that server may be lost.